The patient, AI, and informed consent

Informed consent, a cornerstone of patient autonomy and ethical medical practice, must adapt to accommodate the nuances of artificial intelligence (AI) driven interventions.

By examining the interplay between patient rights, technological advancements, and ethical considerations, there is a clear need for robust consent and preference management systems capable of handling the complexities of AI.

These systems empower patients with granular control over their data and preferences, ensuring transparency, trust, and compliance with legal standards.

As AI continues to reshape healthcare, the principles outlined here provide a framework for integrating AI into patient care pathways, balancing innovation with the fundamental rights of patients.

7 key proposals for informed consent and AI

A recent Frontiers article looks in detail at this concept, ‘The ménage à trois of healthcare: the actors in after-AI era under patient consent’, with 7 key proposals for informed consent in the era of triangular therapeutic alliance between physician, patient, and artificial intelligence:

1. The patient, consistent with the nature of “black boxes,” needs to understand what AI is and how it works.
2. The possibility of withdrawal of consent at any time and optimal privacy management must be guaranteed; the data used must not be traceable to the patient unless explicitly requested by the patient.
3. It must be defined in which nodes AI intervention is proposed, and the patient must be able to choose in which of these to accept or reject it.
4. The role of AI in each individual node must be identified, breaking it down into types of activities performed and level of autonomy in managing them.
5. The consequences of accepting or rejecting AI in each individual treatment step must be made explicit.
6. During each medical act, the patient should be accompanied, explaining to him which activities are performed by the AI and which by the Physician, as well as their respective roles.
7. Adequately trained individuals should be provided to cooperate in drafting and administering consent, technical-procedural explanation, as well as lending assistance in case of ethical dilemmas.

The “black box” challenge of AI

The “black box” challenge in AI refers to the difficulty in understanding and explaining how AI systems, particularly those using deep learning, make their decisions. Here are some key points to help explain this concept:

Complexity of algorithms:

• AI models, especially deep learning models, use complex algorithms that process vast amounts of data to make predictions or decisions.
• These algorithms are often so intricate that even the developers who created them cannot fully understand how they work internally.

Lack of transparency:

• The term “black box” implies that the internal workings of the AI are opaque or hidden from view.
• This lack of transparency makes it challenging to trace how specific inputs lead to particular outputs.

Unpredictable outcomes:

• Because the decision-making process is not fully understood, AI systems can sometimes produce unexpected or incorrect results.
• This unpredictability can be problematic, especially in critical fields like healthcare, where decisions can significantly impact patient outcomes.

Why is the “black box” AI a problem?

Accountability: When AI systems make decisions, it can be difficult to determine who is responsible for those decisions, especially if something goes wrong. This lack of accountability can lead to ethical and legal challenges.

Trust: Users may find it hard to trust AI systems if they do not understand how decisions are made. Building trust is crucial, particularly in sensitive areas like healthcare, where patients need to feel confident in the technology used for their treatment.

Bias and fairness: AI systems can inadvertently learn and perpetuate biases present in the training data. Without transparency, it is challenging to identify and correct these biases, potentially leading to unfair or discriminatory outcomes.

Addressing the black box problem

Explainable AI (XAI):

• Researchers are working on developing AI models that are more interpretable and transparent.
• Explainable AI aims to make the decision-making process of AI systems understandable to humans.

Regulations and standards:

• Implementing regulations that require transparency and accountability in AI systems can help mitigate the black box problem.
• Standards for AI development and deployment can ensure that systems are designed with explainability in mind.

Human-AI collaboration:

• Combining human expertise with AI can help bridge the gap between complex algorithms and understandable decisions.
• Human oversight can ensure that AI systems are used responsibly and ethically.

The significance of informed consent

Informed consent is crucial as it combines the right to health and the right to self-determination. Over time, the relationship between doctors and patients has evolved from a paternalistic model to a collaborative one, where patients actively participate in decision-making.

For informed consent to be valid, it must be:

Personalized: Tailored to the specific patient’s situation.

Voluntary: Patients must freely accept or reject treatment.

Comprehensive: Information provided must be complete and up-to-date.

Understandable: Information must be easy for patients to understand.

Revocable: Patients can withdraw consent at any time.

Key points to consider for consent management with AI

Understanding AI: Patients need to understand what AI is and how it works.

Withdrawal and privacy: Patients must be able to withdraw consent at any time, and their data should remain private unless they choose otherwise.

Choice at each step: Patients should be able to decide at each step of their care whether to use AI.

Role of AI: Clearly define AI’s role in each part of the care process.

Consequences: Explain the consequences of accepting or rejecting AI at each step.

Accompaniment: Patients should be guided through the process, with clear explanations of AI’s and the physician’s roles.

Support: Provide trained individuals to help with consent, explanations, and ethical dilemmas.

How Consent Management can help manage AI

Granular control

Detailed choices: Patients can decide exactly which parts of their data can be used and by whom. For example, they can allow their doctor to see their full medical history but limit a researcher’s access to only certain data points.

Step-by-step decisions: Patients can choose whether to use AI at each stage of their treatment, such as diagnosis, treatment, or follow-up.

Privacy and security

Data protection: Consent management ensures that only authorized people can access patient data, protecting it from misuse.

Transparency: Patients are informed about who is accessing their data and for what purpose, building trust.

Flexibility

Change of mind: Patients can withdraw their consent at any time if they feel uncomfortable with how their data is being used.

Custom preferences: Patients can set preferences on how their data is used, such as for research, treatment, or both.

Compliance with regulations

Legal requirements: Consent management helps healthcare providers comply with laws that protect patient data, like GDPR and HIPAA.

Accountability: Systems keep audit trails and records of who accessed the data and when, ensuring compliance and accountability.